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Three papers from conferences on statewide 
coordination in postsecondary education are presented as part of the 
Inservice Education Program of the Education Commission of the 
States- In "educational Program Budgeting in Oklahoma," Edwaro J. 
Coyle and Dan S- Hobbs outline the principles, procedures, and 
processes utilized by the state in the development of institutional 
needs for educational and general funds. The new budgeting by 
educational program system in Oklahoma is concerned ' with instruction, 
research, and extension. It is suggested that state-level budgeting 
by educational program is superior to the old system of budgeting by 
educational function in that it is more responsive to changing 
conditions; it ties program approval directly to the 
budget-development process; and it is more readily communicated to 
the legislature and the public than the previous system. The second ^ 
conference paper by Robert Befdahl discusses accountability in public 
higher education. It is suggested that a Legislative Program 
Evaluation movement and sunset legislation (to evaluate and either 
terminate or continue state agencies) appear to indicate that the 
state governments may want to evaluate how good a job educational 
institutions are doing (output evaluation) . The third conference by 
Thurston E. Manning considers program evaluation. Evaluations involve 
judgments about the effectiveness and efficiencies of institutions or 
programs, and about the improvement of institutional or program 
effectiveness or efficiency. It is suggested that there is a need for 
states to develop clearer statements of purpose for institutions, 
especially the public institutions and to have a broad base for 
making judgments about educational quality. (SW) 
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Accountability 

by Robert Berdahl 



It is quite remarkable that when the states began 
to create state higher education institutions, they 
did not turn to the normal forms of accountabil- 
ity for running them. Instead, they accepted the 
precedent of the private college board of trustees 
with its corporate form of government. Earl 
Cheit has called this a self-denying ordinance 
whereby the state gave to the corporate boards 
established for public colleges certain privileges 
and immunities not granted to other state activ- 
ities: the right to sue and be sued, to have control 
over their internal finances, to handle personnel 
policies free of state civil service rules and so on. 

Over time, pressures for increased accountability 
have lessened the areas of discretion those corpor- 
ate boards have enjoyed (particularly in areas like 
budgets or determination of new buildings); items 
such as program control, admission of particular 
students, and the hiring, promotion and tenure of 
particular faculty members have always been off- 
limits or beyond the reach of state accountability. 
Although the self-denying ordinance has thus 
been modified with the emergence of statewide 
boards, executive and legislative staffs, PPBS, 
cost benefit anaiysis and better information sys- 
tems, until recently the heart of academe had 
remained beyond the reach. 

Lately, however, there have been some general 
developments in state government which may 
operate ultimately to alter the exemptions from 
traditional state accountability patterns. In the 
executive branch of state government, for in- 
stance, many states changed two-year guberna- 
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torial terms to four years. Many states also re- 
moved the prohibition against reelection of their 
governor and gave the governor item veto powers 
so that he or she could reach inside the budget 
and literally blue pencil individual items and leave 
the rest of the bill intact. 

Occasionally states also reorganized some of their 
floating multimember commissions with staggered 
terms in order to bring them more closely under 
the control of a single governor during his admin- 
istration. Many states put statewide elective offi- 
ces like attorney general, secretary of state, lieu- 
tenant governor on a ticket together or made 
some of them appointive by the elected governor 
to strengthen the coherence of the executive 
branch. 

While all of that was going on, state legislatures 
were not remaining passive in the face of this 
strengthening of the executive branch. They im- 
proved their, staffing and fought for control of 
the postaudit function. !n the past twenty years, 
the postaudit function in over thirty-five states 
has been brought into the legislative branch and 
its activities often broadened in scope and nature 
to go beyond the traditional fiscal and manage- 
ment audits wherein legality and efficiency of 
expenditures were examined. The broadened 
activities have moved to something called per- 
formance or program audits in which the effec- 
tiveness rather than the efficiency of the object 
is examined. Now, it sounds easy to judge wheth- 
er an activity is effective or not. But this implies 
that somebody has an idea somewhere as to what 
ought to be, in normative terms, the output of 
the agency or function jn question. 

The state audit staffs, many of whom are very 
bright young men and women coming out of 
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graduate schools of public policy at flagship 
campuses, join the ranks of the legislative staff 
at the state capital and come back to their alma 
mater in a mood to bite the hand that intellectu- 
ally fed them. They ask some very awkward ques- 
tions and those in higher education do not seem 
very ready or able to offer answers. 

There has emerged in the past few years a move : 
ment called the Legislative Program Evaluation 
(LPE). Rutgers has a clearinghouse for the organi- 
zation at the Eagleton Institute of Politics, and 
the National Conference on State Legislatures in 
Denver is used for administrative coordination. 
The LPE staff wants to know the answers to some 
very stark questions: What are your purposes? 
How do they jibe with legislative intent? To what 
extent is year agency doing what the legislature 
meant for you to do?. How good is it? Could it 
be improved? 

Wherever it is and under whatever forms it takes, 
LPE is coming, and it is no longer granting ex- 
emption and immunity to higher education. In- 
stitutions can not tell them to go away simply 
because the heart of academe has always been 
deferred to by the state accountability pattern. 
Case studies indicate that these staffs are not 
deferring to the professionals in education any 
longer but are demanding the same accountability 
expected from highways, prisons, . agriculture, 
recreation and everybody else. The. plea that high- 
er education is different because of notions of 
academic freedom or .academic institutional 
autonomy just does not seem to go very far. 

Most states are now using LPE studies to under- 
take relatively technical pieces of analysis. There 
are not many states that yet seem ready to aim 
that performance audit weapon at academic pro- 
grams on the four-year campuses. For the mo- 
ment, there is a holding pattern where academic 
programs in most states have not yet been eval- 
uated by the LPE staff. But one senses that it is 
an uneasy plateau period.. 

Alongside is coming another movement, sunset 
legislation, that lends reason to fear that it may 
join with LPE and together become an irresistible 
force. In contrast to the assumption that any 
state agency continues unless it is explicitly ter- 



minated, sunset legislation means that all state 
agencies will be terminated unless explicitly con- 
tinued. Legislators build in a timetable for the 
review and evaluation of the effectiveness of the 
agencies concerned. 

There are three kinds of sunset laws. Some apply 
only to regulatory boards like boards of cosme- 
tology or barbers. There are also selective laws 
that go beyond the regulatory function but stop 
short of being comprehensive. In the third cate- 
gory are the comprehensive laws that say that 
every state agency between now and 1982 will be 
lined up in a certain sequence and will be eval- 
uated and either continued or terminated. One 
does not know what kinds of state staffs will try 
to make these terminal judgments or what kinds 
of criteria they will use as to whether the higher 
education agencies are "effective" or not. 

The Legislative Program Evaluation .movement 
and sunset legislation lead one to believe that the 
people in the state capitol are no longer going to 
be content merely to evaluate inputs: how many 
students, how many faculty, how many square 
feet of buildings and how many dollars. Evalu- 
ators will come around to the exit door and will 
ask how good institutions are at what they are 
doing. This trend is reinforced by the demand for 
minimum competency in elementary-secondary 
education. The day may come when the same 
stringent review will be applied to higher educa- 
tion. 

I would say that those in higher education have 
about ten years to decide this kind of policy 
dilemma of whether to cooperate with the "tech- 
nocrats" in devising systems of evaluation of out- 
put and maybe let them come more quickly or 
to stand in the door and tell them to go away 
since output evaluation is not appropriate to what 
is going on in higher education. Do we cooperate 
with these elements in state government and try 
to make jointly a set of criteria that institutions 
can live with and that do minimum damage to 
what are considered to be academic values? 

The patterns of accountability in the future will 
have to. be much more concerned with the forms 
of evaluation. Statewide boards are now doing 
quite well at the program level, particularly in the 
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approval of new programs. Statewide boards are 
also getting more thoroughly into a review of 
existing programs which, of course, is much more 
controversial. But whether they will go beyond 
that and will start trying to evaluate the outputs 
of higher education is very problematical. If a 
statewide board does not try to help the institu- 



tion tc be its better self in the face of this outside 
demand for performance accountability, I think 
the task will go by default to people, perhaps of 
goodwill and benign intent, who will make their 
own systems of accountability, and academe may 
have to live in a straitjacket that is not one of 
its own making. 
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Educational Program Budgeting in Oklahoma 

By 

Edward J. Coyle and Dan S. Hobbs 

Oklahoma Stc*® Regents for Higher Education 



The Oklahoma State System of Higher Education 
is made up of 27 colleges and universities plus 
eight other constituent agencies whose functions 
are related to public higher education. These in- 
stitutions and agencies currently enroll 127,536 
students and will expend $219,866,634 for 
educational and general purposes during 1976-77. 
This figure does not include $37.7 million to be 
expended from sponsored research and other 
sponsored program funds. 

Article Xlll-A of the Oklahoma Constitution 
provides that the Oklahoma State Regents for 
Higher Education shall constitute a coordinating 
board of control for all State System institutions 
and agencies, whose specified powers include (a) 
the prescribing of standards of higher education 
for each institution, (b) the determination of 
functions and courses of study in each of the in- 
stitutions to conform to the standards prescribed, 
(c) the granting of degrees and other forms of 
academic recognition for completion of prescribed 
courses, and (d) recommending to the Oklahoma 
Legislature the budget allocations to each in- 
stitution. 

Ideally, a state-level system to determine the 
budgetary needs of colleges and universities 
should meet three basic objectives. The system 

^ should provide for adequacy, equity, and ac- 
countability. Each institution or agency needs a 

* budget allocation sufficient to carry out its 
assigned functions and programs. Likewise, in- 
stitutions should have the perception that they 
have been treated fairly in comparison with other 
institutions. Thirdly, there must be a consensus on 
the part of the Legislature and the people that 
public funds are being wisely allocated and ad- 
ministered. 

A state-level system for determining institutional 
needs should likewise possess some secondary 
characteristics, including the ability to respond to 
changing conditions, either up or down; the 
capability of being readily communicated to in- 



stitutions and the public; and flexibility to provide 
for ease of administration at the campus level. Also, 
it is to be hoped that the data requirements to 
make the system run would not be so great as to 
outweigh its benefits in the form of analytical data 
to be used by administrators in the evaluation of 
institutional performance. 

Development of Educational Program Budgeting 

Some twenty-five years ago, Oklahoma pioneered 
in the development of a new "formula by budget 
function" system for presenting the needs of 
colleges and universities to the Oklahoma 
Legislature. That is, institutional needs were 
spelled out in terms of how many dollars were 
needed to pay for (1) administration, (2) general 
expense, (3) instruction and departmental 
research, (4) librafry, (5) organized activities, (6) 
organized research, (7) extension and public 
service, and (8) physical plant. 

That system had many advantages, and served 
well during the relatively static days of the 1950's 
and early 1960's. However, it proved to be 
unresponsive to changing societal conditions in. 
that it favored the small inefficient college over the 
growing, more efficient institution. Also, it was 
difficult of communication in dealing with the 
Oklahoma Legislature. It was therefore determined 
that a new system was needed to see institutions 
through the rapidly changing growth years of the 
late 1960's and 1970's. 

The new "budgeting by educational program" 
system in Oklahoma is akin to, but not identical 
with, the system being developed by the NCHEMS 
organization. It is predicated on the assumption 
that ali institutions carry out three primary func- 
tions or activities, namely (1) the transmission of 
higher knowledge (Instruction), (2) the 
development of new knowledge (Research) and 
(3) the extension of higher education programs 
and services to the public at large (Extension and 
Public Service). 
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For educational program budgeting purposes, 
the three primary categories of Instruction, 
Research and Extension are divided into sub- 
categories called programs, research programs, 
and extension programs. To this date, only the 
functional area of instruction has been developed 
systematically on a program-by-program basis. 
Next in the list of items to be accomplished is the 
functional area of extension, to be followed by 
research. 

During the early 1960's the State Regents began 
to gather and store various kinds of statistical data 
in preparation for budgeting by educational 
program. These data included such items as 
student enrollment by level, instructional salaries 
of faculty, the cost of teaching individual courses 
and educational programs, and the like. There is 
now on file at the state level a decade or more of 
such historical data. 

The new educational program budgeting system 
was first utilized on a trial basis with one in- 
stitution some five years ago. The following year, 
1973-74, the new system was used for three in- 
stitutions representing three different institutional 
types. In 1974-75, the new system was generalized 
to include all institutions. 

Although it is probably too early to assess the 
effectiveness of the new budget-needs system 
after only a three-year trial, few preliminary ob- 
servations can be made at the current stage of 
development. First, state-level budgeting by 
educational program is superior to the old system 
of budgeting by educational function in that it is 
more responsive to changing conditions; it is more 
rational than the old system for the reason tnat it 
ties program approval directly to the budget- 
development, process; and it is more readily 
communicated to the Legislature and the public 
than the previous system. It also spills out better 
analytical data for institutional use than the budget 
function method. 

The limitations of the new system are that it 
requires a somewhat greater amount of data to run 
than did the older system, and the data outputs are 
not directly related to the way institutions budget 
at the campus level. It may therefore be concluded 
that budgeting by educational program is a more 
valuable state-level tool than an institutional tool 
at this point. Overall, however, the Oklahoma 
experience with educational program budgeting 
has been a salutary one. 

The remainder of this paper will be devoted to a 



detailed description of the principles, procedures 
and processes utilized by the Oklahoma State 
Regents for Higher Education in the development 
of institutional needs for educational and general 
funds by the educational program budgeting 
method. 

Background 

One of the responsibilities of the Oklahoma State 
Regents for Higher Education, as set out Jn Section 
2, Article Xlll-A of the Constitution of Oklahoma, is 
to "...recommend to the State Legislature the 
budget allocations for each institution." In 
discharging this constitutional responsibility, with 
respect to operating budget n?eds, the State 
Regents prepare and submit to each Oklahoma 
Legislature a recommendation for appropriation oi 
funds for the institutions in The Oklahoma State 
System of Higher Education. The recommendation 
is developed and presented for each institution on 
a "Program Budget" basis which identifies each 
program workload by level in terms of full-time 
equivalent (FTE) students for each of the 
programs together with the anticipated dollar need 
per FTE for each of the levels for each program, the 
total needs for each program and the total needs 
for the educational programs of the institution. 

Preparation of the budget needs entails con- 
siderable research and study on the part of the 
State Regents. In arriving at th needs of in- 
stitutions, it is necessary to gather and consider 
information about such factors as functions and 
educational programs of institutions, the student 
enrollment of institutions, faculty and staff man- 
power requirements, faculty salaries, and the like. 
Also, the State Regents conduct study sessions 
with presidents of colleges and universities in the 
State System to obtain their views as to the needs 
of their respective institutions. 

Criteria 

In order to assist the State Regents in applying 
the results of their research and study to ac- 
complish their constitutional and statutory 
responsibilities, they adopt Guiding Principles and 
steps of Procedures. While these may be modified 
from time to time to accommodate changing 
policies, they provide reliable reference points for 
understanding the program budgeting system and 
for evaluating the performance of the system. For 
the fiscal year 1977-78 principles and procedures 
have been adopted as follows: 



Guiding Principles 

1. Oklahoma should support the educational 
programs of institutions in the State System at 
such level that will provide high-quality 
educational performance. 

2. Each institution in the State System should 
carry out three broad areas of educational program 
responsibility: (a) instruction, (b) research, and 
(c) public service. 

3. The budget needs of each institution should be 
determined on the basis of educational program 
costs including the three categories above 
projected for the budget year. 

4. The instructional program costs should in- 
clude: (a) resident instruction, (b) orgainzed 
activities related to instruction, (c) library, (d) 
general administration, (e) general expense, and 
(f) operation and maintenance of the physical 
plant. Budget needs for organized research and 
extension and public service should be computed 
separately from instructional costs. 

5. Primary factors that should be recognized in 
determining the instructional part of the budget 
needs should include: (a) type of institution, (b) 
instructional program costs by level, and (c) full- 
time-equivalent enrollment by level for the full 
fiscal year. 

6. Operating experience of institutions in 
Oklahoma and institutions in the 10-state region 
including (a) ratios of students to faculty by level, 
(b) average salaries of full-time faculty by type of 
institution, and (c) total educational and general 
budget student per capita expenditure, should be 
recognized as influence factors when establishing 
instructional program costs. 

7. The amount of funds to be added to the budget 
for research is then determined. Basic functions of 
institutions will, of course, influence the deter- 
mination of these amounts. Institutions that ac- 
complish significant research must be provided 
funds accordingly. All institutions carry on some 
type of institutional research and study of 
programming and thus must be provided an 
allowance for this function of the budget. 

8. Extension and public service is the third 
category of the educational program budget. Again, 
institutions' functional assignments will influence 
the degree to which they participate in extension 



programs and/or public service programs. An 
appropriate amount for this program is accordingly 
determined. 

9. The final step is to add the amounts deter- 
mined for (a) instructional programs, (b) research, 
and (c) extension and public service. This becomes 
the total amount of budget needs of the institution 
for the educational programs to be operated. 

Funding 

An estimate is made of the amount of income 
expected to be available during the budget year 
from Revolving Funds. This is subtracted from the 
total institutional budget requirement for the year 
and the difference becomes the amount requested 
for State-Appropriated Funds. 

"State-Appropriated Funds" are those funds 
appropriated by the Legislature to the State 
Regents to be allocated to constituent institutions 
and agencies. "Revolving Funds" are those funds 
which the institutions receive from student fees, 
sales and services of educational departments, the 
Federal Government in some instances, and from 
other miscellaneous sources. The amount of 
"Revolving Funds" income is estimated for each 
institution after taking into consideration its fee 
schedule, the function of the institution and 
possible charges to be made by the institution for 
the different services rendered, and the past ex- 
perience of the institution as to funds actually 
received from the various sources. 

Results 

Program budgeting results for twenty-five in- 
stitutions in The Oklahoma State System of Higher 
Education for 1977-78 show a projected 104,620 
FTE students will be served in 1,725 programs at a 
total needs of $182,186,944. Perhaps even more 
significant from a public policy standpoint, these 
results include (1) the program production or 
output emphasis which provides the Legislature 
and the Governor with a better understanding of 
what the State is to receive for the money 
requested; (2) a firm basis for equitable allocation 
of funds among institutions; and (3) criteria for 
accountability of institutions and the State System 
in accomplishing the work proposed. 
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Program Evaluation 

by Thurston E. Manning 



Evaluation of quality of institutions or programs 
is in most states a responsibility of individual in- 
stitutions acting alone or through the voluntary 
accreditation process. A fundamental point is 
' that judgment about quality must be based on an 
understanding of purpose: unless one is sure what 
purpose an institution or program is to serve one 
cannot judge whether it is achieving the purpose. 
And the degree of success in achieving educational 
purpose is the measure of educational quality. 

Evaluation itself can serve different ends, and the 
form of evaluation must be chosen with this in 
mind. Among the ends of evaluation are judg- 
ments about (a) the effectiveness of institutions 
or programs (that is, are they in fact achieving 
their purposes); (b) the efficiencies of institutions 
or programs (that is, are they achieving their pur- 
poses at reasonable cost); and (c) the improvement 
of institutional or program effectiveness or ef- 
ficiency. The accreditation process has taken as 
a primary end the improvement of institutions 
and programs. As a result its procedures are dif- 
ferent from those that would be used if a judg- 
ment of effectiveness or efficiency were the only 
outcome of the evaluation. Indeed, for many in- 
stitutions the end of improvement is more im- 
portant than the effectiveness judgment, and 
that judgment is therefore an incidental outcome. 

The future concerns of states in assuring institu- 
tional and program quality, require addressing 
several issues. First of these is the need for states 
to develop clearer statements of purpose for insti- 
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tutions, especially the public institutions. Institu- 
tional statements of purpose are often vague and 
almost theological in language. What is needed are 
statements sufficiently precise to gauge whether 
the activities of the institutions do- achieve the 
purposes — the measure of institutional quality. 

A second issue is the inherent conflict between 
central control of a system of institutions and the 
local autonomy of each institution. States need to 
find a middle ground that will allow for sufficient 
coherence of the system to assure statewide ob- 
jectives, while permitting the flexibility that will 
permit institutions to respond to new local de- 
mands and engage in constructive innovation. A 
system that has too tight central control is inflex- 
ible and unresponsive; a system that is completely 
decentralized lacks coherence, tends to overlook 
statewide concerns, and frequently has unneeded 
duplications. The balance between the two ex- 
tremes is a day-by-day political problem having 
solutions unique in each state. 

The third issue is the need for states to have a 
broad base for making judgments about educa- 
tional quality. Intrastate conflicts among institu- 
tions do exist and may affect judgments about in- 
stitutions. Particularly serious is the situation in 
which the governing board of a public institution 
becomes the state licensing agency for private in- 
stitutions. In such cases, a judgment restraining or 
prohibiting the operations of a private institution 
is often seen as arising from a fear of competition 
on the part of the licensing agency which is simul- 
taneously an institution competing for students. 

But perhaps a more important reason for a broad 
base of judgment is the fact that no state's educa- 
tional system stands on its own. Consequently 
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there is a need to look beyond the borders of a 
state, to "be aware of the interstate effects of 
educational decisions and judgments, and to util- 
ize knowledge and experience available beyond 
the state borders. 

A possible way for states to obtain a broader base 
for judgments of educational quality of its institu- 
tions is to utilize more fully the activities of the 
voluntary accrediting associations. These organi- 
zations extend over several states, thus providing 
a geographical broadening, and in addition in- 
corporate both public and private institutions. 
While accreditation currently focuses on indivi- 
dual institutions or programs (and thus does not 
deal directly with the state concern of coordina- 
tion among institutions), it represents an accepted 
means by which judgments about the quality of 
those institutions and programs can be made. 

Two final comments. First, while it is true that 
educational quality means the successful meeting 



of valid educational purposes, the .states should be 
aware that their educational systems have other 
effects. From time to time a state should conduct 
what Scriven has called a "goal-free" evaluation: 
simply having qualified persons examine the ef- 
fects of an institution or program without refer- 
ence to the purposes. In this way important out- 
comes unrelated to stated purposes may be 
discovered. 

Second, states — or more exactly the people of 
a state — should have some tolerance for the mis- 
takes that will be made in any human enterprise. 
Public organizations often seem so intolerant of 
error that primary attention is given to error 
avoidance. But if the educational enterprise is to 
be flexible enough to meet the new and develop- 
ing needs of society it must encourage trying new 
things. Some of these trials will not be successful; 
mistakes will occur. Never committing mistakes 
means never doing anything new and untried. The 
cost of mistakes is less than the cost of inaction. 
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